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Advanced  Telepresence  Surgery  System 
Development 


Joel  F-  Jensen 
John  W.  Hill 


Medical  Technology  Laboratory 
SRI  International 
Menlo  Park,  CA  94025 


ABSTRACT 

SRI  International  is  currently  developing  a  prototype  remote  telepresence 
surgery  system,  for  the  Advanced  Research  Projects  Agency  (ARPA),  that  will  bring  life¬ 
saving  surgical  care  to  wounded  soldiers  in  the  zone  of  combat.  Remote  surgery  also  has 
potentially  important  applications  in  civilian  medicine.  In  addition,  telepresence  will  find 
wide  medical  use  in  local  surgery,  in  endoscopic,  laparoscopic,  and  microsurgery 
applications.  Key  elements  of  the  telepresence  technology  now  being  developed  for 
ARPA,  including  the  telepresence  surgeon’s  workstation  (TSW)  and  associated  servo 
control  systems,  will  have  direct  application  to  these  areas  of  minimally  invasive  surgery. 
The  TSW  technology  will  also  find  use  in  surgical  training,  where  it  will  provide  an 
immersive  visual  and  haptic  interface  for  interaction  with  computer-based  anatomical 
models.  In  this  paper,  we  discuss  our  ongoing  development  of  the  MEDFAST 
telesurgery  system,  focusing  on  the  TSW  man-machine  interface  and  its  associated  servo 
control  electronics. 
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INTRODUCTION 

SRI  International  is  currently  developing  a  prototype  remote  telepresence 
surgery  system,  for  the  Advanced  Research  Projects  Agency  (ARPA),  that  will  bring  life¬ 
saving  surgical  care  to  critically  wounded  soldiers  in  the  combat  zone  (CZ).  Using  SRI’s 
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telepresence  technology,  a  surgeon  located  in  a  mobile  army  surgical  hospital  (MASH) 
unit  or  base  hospital  will  carry  out  emergency  surgical  procedures  on  soldiers  in  an 
armored  mobile  surgical  vehicle  in  the  CZ.  Dubbed  the  Medical  Emergency  Forward 
Area  Surgical  Telepresence  (MEDFAST)  system,  this  vehicle  will  contain  a  fully 
equipped  telepresence  operating  theater,  and  a  medic  in  the  MEDFAST  will  assist  the 
remote  surgeon. 

Remote  telepresence  surgery  also  has  applications  in  civilian  medicine,  where  it 
will  enable  specialists  in  regional  medical  centers  to  treat  patients  in  outlying  clinics.  In 
addition,  telepresence  will  most  likely  find  wide  medical  use  in  local  surgery — in 
endoscopic,  laparoscopic,  and  microsurgery  applications,  and  will  play  an  important  role 
in  surgical  training,  providing  an  immersive  visual  and  haptic  interface  to  computer- 
based  anatomical  models. 

In  current  practice,  microsurgery  involves  delicate  manipulation  of  small  tissue 
structures  viewed  through  a  stereo  microscope.  While  major  reconstructive  efforts  such 
as  reattachment  of  severed  digits  are  routinely  performed,  these  microsurgical  procedures 
are  often  long  and  exhausting,  with  the  surgeon  spending  many  hours  of  concentrated 
work  looking  through  microscope  eyepieces,  sitting  in  an  awkward  position  [Franken  et 
al.,  1995].  In  addition,  human  performance  limitations  preclude  the  extension  of 
microsurgical  techniques  to  new  applications  using  higher  magnification.  The  time  to 
complete  a  task  increases  in  proportion  to  the  visual  magnification  required  to  complete  it 
[Szabo,  1994].  Therefore,  increases  in  magnification  result  in  longer  procedures,  which 
increase  patient  risk  and  incur  greater  financial  costs. 

Similar  limitations  exist  for  laparoscopic  surgery  as  currently  practiced.  It  has 
largely  replaced  open  surgery  for  relatively  simple  procedures  such  as  cholecystectomy 
and  hernia  repair,  and  brings  substantial  benefits  to  the  patient,  including  shorter  hospital 
stay,  faster  recovery,  and  less  cosmetic  damage.  However,  with  current  instrumentation, 
laparoscopic  procedures  require  more  operating  room  time  and  are  much  more  difficult 
for  the  surgeon  than  equivalent  open  procedures.  As  a  result,  relatively  few  surgeons  will 
be  able  to  progress  fi-ora  laparoscopic  cholecystectomy  to  more  complex  procedures  in 
the  bowel  and  esophagus.  The  problem:  laparoscopic  surgery  currently  entails 
awkwardly  maneuvering  long  instruments  that  are  fiilcrumed  in  the  patient’s  abdominal 
wall  (so  that  the  motion  of  the  tool  tips  is  opposite  to  the  motions  of  the  surgeon’s  hands), 
while  viewing  the  surgical  field  on  a  2D  video  display  located  on  a  stand  above  the 
patient.  This  arrangement  makes  hand-eye  coordination  very  difficult,  and  provides  the 
surgeon  with  very  little  force  feedback. 

SRI’s  telepresence  technology  can  overcome  many  of  the  limitations  imposed 
by  current  minimally  invasive  surgery  (MIS)  instrumentation  and  methods.  By  enabling 
a  great  increase  in  the  range  and  number  of  procedures  that  can  be  performed, 
telepresence  will  bring  the  benefits  of  MIS  to  many  more  patients. 

In  the  case  of  microsurgery,  telepresence  systems  will  provide  motion  and  visual 
scaling  along  with  amplified  force  feedback  that  will  allow  surgeons  to  perform 
microsurgical  procedures  with  the  same  speed,  skill,  and  dexterity  they  achieve  in  full- 
scale  open  surgery.  Similarly,  for  laparoscopic  and  other  endoscopic  procedures. 


telepresence  systems  will  provide  an  intuitive  and  effective  interface  for  telepresence 
surgery  that  will  greatly  enhance  surgeons’  capabilities. 

Key  elements  of  the  MEDFAST  system  we  are  now  developing,  including  the 
telepresence  surgeon’s  workstation  (TSW)  and  associated  servo  electronics,  will  find 
direct  use  in  these  MIS  telepresence  systems  and  will  play  an  important  role  in  future 
MIS  practice.  In  addition,  the  new  TSW  will  be  useful  in  surgical  training,  providing  an 
excellent  visual  and  haptic  interface  for  interaction  with  computer-based  anatomical 
models. 


In  this  paper  we  discuss  our  ongoing  development  of  the  MEDFAST  system, 
focusing  on  the  TSW  man-machine  interface  and  its  associated  servo  control  electronics. 


BACKGROUND 

SRI  International  has  previously  developed  and  demonstrated  a  prototype 
telepresence  surgery  system  [Green  et  al.,  1995]  that  has  been  used  to  perform  successful 
remote  surgical  procedures  on  animals.  Other  robotic  and  telerobotic  systems  have  been 
developed  for  surgical  applications  [Burckhardt  et  al.,  1995;  Taylor  et  al.,  1994],  but  only 
the  SRI  system  provides  the  combination  of  responsive,  force-reflecting  telemanipulators 
and  integrated  visual  feedback  that  allows  surgeons  to  remotely  perform  the  full  spectrum 
of  surgical  maneuvers — for  example,  cutting,  suturing,  dissecting — ^with  a  high  degree  of 
dexterity  and  effectiveness  comparable  to  their  hands-on  skills. 

The  SRI  telepresence  technology  is  an  entirely  new  method  of  remote 
manipulation,  with  important  potential  applications  in  remote  surgery  and  MIS. 

Surgeons  who  have  worked  with  SRI’s  telepresence  surgery  demonstration  system,  with 
its  fully  integrated  combination  of  three-dimensional  (3D)  video  imaging,  precise  remote 
manipulation  with  handheld  surgical  instruments,  and  force  feedback,  agree  that  it  is  so 
compellingly  realistic  as  to  immediately  dispel  any  sense  of  remote  operation.  Using  the 
demonstration  system,  these  surgeons  have  successfully  performed  three  series  of  remote 
surgical  procedures  on  pigs  [Bowersox  et  al.,  1996].  These  first-ever  remote  surgeries 
included  treatment  of  a  variety  of  simulated  abdominal  and  vascular  injuries,  including 
cholecystectomy,  nephrectomy,  repair  of  gastrotomy  and  enterotomy,  and  repair  of 
femoral  artery,  liver,  and  bladder  lacerations. 

Shown  in  Figure  1,  the  demonstration  system  consists  of  two  modules:  a  work¬ 
site  module,  where  the  actual  object  manipulation  takes  place,  and  an  operator  module 
that  contains  a  strikingly  realistic,  3D,  video  reproduction  of  the  actual  work  space  (not  a 
computer  model).  This  demonstration  system  incorporates  many  advanced  technologies, 
including  a  highly  responsive,  force-reflecting  master/slave  manipulator  with  4  degrees 
of  freedom  (DOF),  stereoscopic  video,  and  stereophonic  sound.  In  effect,  the  operator 
grasps  surgical  instruments,  reaches  into  what  appears  to  be  the  actual  workspace,  and 
carries  out  complex  tasks  with  quick,  sure  motions.  Operating  surgeons  feel  as  though 
they  are  performing  tasks  right  before  their  own  eyes.  The  3D  image  is  created  by  a 
stereographic  video  system,  viewed  with  a  mirror.  The  surgeon  looks  downward, 
“through”  the  mirror,  to  see  the  surgical  site  below  it,  just  where  he  or  she  is  reaching 


(a)  The  telepresence  surgeon  ’  s 

workstation  of  the  demonstration 
system  provides  two  hand-control 
masters  with  surgical  instrument 
handles,  a  stereographic  view  of 
the  surgical  field,  and  a 
microphone  and  speakers  for 
communicating  with  the  assistant 
at  the  remote  site. 

Figure  1.  Demonstration  telepresence  surgery  system 

with  the  hand  controls.  The  control  is  a  surgical-instrument  handle  mounted  on  a  light, 
well-balanced,  force-reflecting  servo  manipulator — ^the  master.  An  identical  slave  servo 
manipulator  at  the  operating  table  controls  the  actual  surgical  instrument — ^scalpel, 
forceps,  and  so  forth — the  tip  of  which  is  seen  in  the  image,  as  if  it  were  actually  attached 
to  the  handle.  When  the  tip  touches  the  tissue,  the  resistance  is  felt  from  the  handle.  The 
overall  effect  is  very  compelling.  With  support  from  the  National  Institutes  of  Health, 
ARP  A,  and  other  sources,  SRI  has  continued  the  development  of  this  technology  for 
remote  surgery  as  well  as  laparoscopic  and  microsurgery  applications. 


(b)  The  demonstration  system  remote 
surgery  unit  comprises  two  slave 
manipulators  with  interchangeable 
surgical  instruments,  a  pair 
of  stereographic  video  cameras, 
and  stereo  microphones. 


THE  ADVANCED  SURGERY  SYSTEM 


System  Block  Diagram 

•  A  simplified  block  diagram  of  the  advanced  system  is  shown  in  Figure  2. 

We  have  established  specifications  for  the  master  and  slave 
telemanipulators  and  the  stereo  video  display.  These  specifications,  which 
form  the  basis  for  the  TSW  design,  are  summarized  below. 


Figure  2.  MEDFAST  system  block  diagram.  The  MEDFAST  system  consists  of  the 
telepresence  surgeon’s  workstation  (TSW),  the  communication  link,  the 
remote  surgery  unit  (RSU),  and  electronics  modules  associated  with  the  TSW 
andRSU. 

•  Control  handles  for  master  manipulators,  interchangeable  for  surgeon’s  use 
with  either  right  or  left  hand 

~  Hemostat  handle 

-  Forceps  (pickup)  handle 

•  Stereo  vision  system 

-  Display  resolution:  640  x  480  pixels,  minimum 

-  Camera  resolution:  700  horizontal  by  494  vertical  lines 

-  Stereo  display  type:  120  Hz  full-frame  with  active  or  passive  glasses 


TheNewTSW 


Based  on  the  above  specifications,  we  are  developing  a  new  TSW  with  two 
6-DOF  (plus  grip)  hand  controls,  enhanced  displays,  auxiliary  controls,  and  all-new, 
ruggedized  electronics.  Using  wood  and  metal  mock-ups,  we  have  had  consulting  *. 
surgeons  perform  simulated  procedures,  including  suturing  organ  models,  to  evaluate  the 
range  of  motions  required  for  surgical  procedures  and  guide  our  designs  as  they  evolve. 
The  surgeons  are  experienced  in  trauma,  general,  and  vascular  surgery,  as  well  as  other 
surgical  specialties. 

i  Master  Manipulator  Design :  The  proposed  manipulator  design  is  shown  in 

Figure  3. 


(a)  Overview  of  (b)  Detail  of  master  with 

manipulator  pickup  handle 


Figure  3.  Design  of  surgical  manipulators 

TSW  Console  Human  Factors:  The  new  surgeon’s  console  has  been  designed 
to  accommodate  surgeons  with  a  wide  range  of  physical  statures.  The  design  will 
comfortably  seat  men  and  women  ranging  from  the  5th  to  the  95th  percentile  in  height, 
and  will  enable  effective  telepresence  for  all  of  them.  Figure  4,  based  on  data  from 
Woodson  and  Conover  [1964],  illustrates  the  latitude  of  adjustment  necessary.  Our  plan 
is  to  have  the  surgeon  sit  on  a  chair  of  adjustable  height,  which  will  bring  his  or  her  eyes 
to  the  proper  location  to  view  the  display.  A  foot  rest  with  foot  controls  will  be  brought 
up  to  match  a  comfortable  foot  height,  and  (if  necessary)  the  master  arms  can  be  adjusted 
up  or  dovm  to  a  comfortable  position. 

To  accommodate  the  6-DOF  manipulators,  the  new  surgeon’s  console  will  need 
to  provide  a  much  larger  workspace  than  is  afforded  by  the  current  console.  Accordingly, 
the  new  console  is  designed  to  allow  unimpeded  motion  of  the  control  handles 
throughout  a  range  of  at  least  1  cubic  foot  directly  in  front  of  the  surgeon,  to  allow  rapid 
and  effective  surgical  treatment  of  trauma.  The  conceptual  design  is  shown  in  Figures  5a, 
5b,  and  5c.  For  finer,  more  controlled  procedures,  such  as  suturing  and  micro  dissecting, 
the  console  will  include  positionable  forearm  rests  for  stabilization. 


Figure  4.  Variation  in  human  dimensions 


(c)  Top  view 

Figure  5.  Design  of  telepresence  surgeon’s  workstation 


Both  the  stereographic  display  and  heads-up  display  are  also  included  in  the 
surgeon’s  workstation.  The  stereo  display  is  viewed  through  a  mirror  as  in  our  initial 
telepresence  demonstration  system.  The  heads-up  display  (not  shown  in  Figure  5)  will  be 
at  the  back  of  the  TSW  console.  Additional  items  not  shown  include  foot  controls,  audio 
microphone,  and  stereo  speakers. 


Servo  Controller 

New  servo  electronics  and  algorithms  have  been  developed  to  control  the 
6-DOF  master  and  slave  manipulators.  To  enable  enhanced  tactile  feedback,  enhanced 
dynamic  response,  and  greater  manipulator  output  force,  we  have  developed  and  tested 
new  control  algorithms  that  employ  a  combination  of  force  and  position  feedback.  The 
algorithms  have  been  tested  using  simulation  and  actual  operation  on  a  single-axis  test 
stand. 


Simulation :  We  have  developed  a  model  of  our  DSP-based  controller  and  a 
motor-driven  test  axis  using  Matlab  and  SIMULINK  (products  of  Math  Works,  Inc.)  to 
aid  in  developing  and  testing  different  bilateral  servo  algorithms.  We  constructed  a 
detailed  analytical  and  numerical  model  of  our  control  hardware  and  the  dynamics  of  the 
servo  control  test  stand  so  as  to  better  understand  the  control  issues  and  to  speed  and 
facilitate  alternative  control  designs.  The  model  includes  discrete  sampling  effects  and 
real-time  computation  delay  effects. 

Servo  Test  Stand :  We  have  built  a  single-axis  servo  test  stand  to  verify  the 
simulation  results.  The  test  stand  consists  of  two  motor-driven  axes  with  gearing  and 
inertia  similar  to  the  elbow  axis  of  the  new  manipulators.  Each  axis  has  a  tachometer  and 
potentiometer  for  determining  step  and  frequency  response.  In  addition,  an  external  drive 
motor  on  one  axis  produces  sinusoidal  input  (simulating  operator  hand  motion)  at 
different  frequencies  for  determining  transfer  functions.  A  force  sensor  on  the  tip  of  each 
arm,  sensitive  to  ±2  lb,  is  interfaced  to  the  analog  inputs  of  our  servo  controller. 

Evaluation  of  Control  Laws:  We  have  evaluated  the  following  known  control 
laws  for  bilateral,  force-reflecting  control  of  teleoperated  manipulators. 

•  Position/position  control :  First  introduced  over  thirty  years  ago  [Burnett, 
1957],  position/position  control  used  bilateral  position  tracking  to  introduce 
some  force  reflection  into  a  teleoperated  manipulator. 

•  Power  steering  and  position/position  control :  Local  power  steering  can 
be  introduced  into  the  position/position  control  scheme  by  feeding  back  a 
force  measurement  to  the  local  controller. 

•  Position/position  control  and  cross-coupled  force  reflection:  Power 
steering  is  made  symmetric  by  passing  force  measurements  between  master 
and  slave. 

•  Position  forward/force  back:  This  is  an  asymmetric  control  objective  that 
has  the  slave  manipulator  follow  the  master  manipulator’s  position  only, 
while  the  master  manipulator  follows  the  slave’s  force  measurement  only 
[Flateau,  1969].  Additional  damping  is  provided  by  some  velocity 
feedback.  This  approach  can  an  be  used  with  a  single  (slave)  force  sensor. 

•  Ideal  kinesthetic  coupling:  A  physically  derived  control  law  using 
acceleration  feed  forward  and  cross-coupled  force  measurements  provides 
‘‘ideal  kinesthetic  coupling”  [Yokokohji  and  Yoshikawa,  1994]. 


We  are  currently  evaluating  each  of  the  control  laws  listed  in  Table  1  with  computer 
models  and  the  single-axis  servo  test  stand. 

Table  1 

KINESTHETIC  COUPLING  CONTROL  LAWS 


Performance 

Passive 

Optimality  and  Tuning 

Bilateral  position 
tracking  (BPT) 

Tracks  position,  force 
tracking  contaminated 
by  manipulator  inertia 

Yes 

No  automatic  tuning, 
suboptimal  performance 

Power  steering 
with  BPT 

Reduced  inertia  compared 
to  BPT  alone,  loss  of 
contact  stiffness 

No 

No  automatic  tuning, 
suboptimal  performance 

Bilateral  force 
amplification  with 
BPT 

Good  stiffness  and  low 
inertia 

No 

No  automatic  tuning, 
suboptimal  performance 

Position  forward/ 
force  back 

Good  stiffness  and  low 
inertia,  can  be  used  with 
single  force  sensor 

No 

No  automatic  tuning, 
suboptimal  performance 

Ideal  kinesthetic 
coupling 

Good  stiffness  and  low 
inertia 

Yes 

Model-based  tuning, 
suboptimal  performance 
(acceleration  filtering) 

New  Control  Electronics:  New  control  electronics  have  been  designed  and 
built  to  accommodate  both  force-  and  position-feedback  signals. 


COMMENTS  AND  CONCLUSIONS 

As  part  of  SRI’s  ongoing  project  to  develop  a  MEDFAST  telesurgery  system  for 
ARPA,  we  are  developing  an  advanced  TSW  design  that  will  enable  highly  effective 
telepresence  for  remote  surgery  as  well  as  MIS  applications,  and  will  also  serve  as  an 
excellent  immersive  interface  for  surgical  training  using  computer-based  anatomical 
models.  The  TSW  will  include  6-DOF  (plus  grip)  force-reflecting  master  manipulators, 
high-resolution  3D  video,  heads-up  peripheral  video  displays,  foot  switches,  and  other 
I/O  devices.  The  new  console  will  allow  unimpeded  motion  of  the  master  control 
handles  throughout  a  range  of  at  least  1  cubic  foot  directly  in  front  of  the  surgeon, 
enabling  rapid  and  effective  surgical  treatment 


ACKNOWLEDGMENTS 

This  work  was  supported  in  part  by  SRI  International  and  in  part  by  the  National 
Institutes  of  Health,  under  Grant  No.  GM  44902,  and  by  the  Advanced  Research  Projects 


Agency  (ARP A)  under  Contract  N00014-94-C-0040,  P.O.  number  32858  from  Assurance 
Technology  Corp.  (Government  Contract  N00014-93-D-2022),  and  Grant  DAMD 17-95- 
1-5019.  The  views  represented  in  this  article  are  those  of  the  authors  and  do  not  reflect  the 
official  policy  of  the  Department  of  Defense  or  the  U.S.  Government  (approved  for 
public  release,  distribution  unlimited). 


REFERENCES 

Bowersox,  J.C.,  A.S.  Shah,  J.F.  Jensen,  J.W.  Hill,  and  P.S.  Green,  “Vascular  Applications 

,  of  Telepresence  Surgery:  Initial  Feasibility  in  Swine,”  J.  Vase.  Surgery,  in  press 
(1996). 

Burckhardt,  C. W.,  P.  Flury,  and  D.  Glauser,  “Stereotactic  Brain  Surgery,”  IEEE  EMBS 
Magazine,  Vol.  14,  No.  3,  pp.  314-316  (May/June  1995). 

Burnett,  J.R.,  “Force-Reflecting  Servos  Add  ‘Feel’  to  Remote  Controls,”  Control 
Engineering,  Vol.  4,  No.  7,  pp.  1269-1274  (1957). 

Flatau,  C.R.,  “Compact  Servo  Master-Salve  Manipulator  with  Optimized  Communication 
Links,”  Proc.  17th  Conf.  Remote  Systems  Technology,  pp.  154-164  (November 
1969). 

Franken,  R.,  S.C.  Gupa,  S.R.  Rod,  S.V.  Thomas,  J.H.  Barker,  M.  Kon,  and  J.C.  Banis: 
“Microsurgery  without  a  Microscope:  Development  of  a  Three-Dimensional  On¬ 
screen  Microsurgery  System  (TOMS),”  J.  Medicine  and  Virtual  Reality,  Vol.  1,  No. 
1,  pp.  26-32  (1995). 

Green,  P.S.,  J.W.  Hill,  J.F.  Jensen,  and  A.  Shah:  “Telepresence  Surgery,”  IEEE  Eng. 
Medicine  and  Biology,  pp.  324-329  (May/June  1995). 

Szabo,  Z.:  Personal  communication.  Director,  MOET,  San  Francisco  (1994). 

Taylor,  R.H.,  et  al.,  “An  Image-Directed  Robotic  System  for  Precise  Orthopaedic 
Surgery,”  IEEE  Trans.  Robotics  and  Automation,  Vol.  10,  No.  3,  pp.  261-265 
(1994). 

Woodson,  W.,  and  D.  Conover,  Human  Engineering  Guide  for  Equipment  Designers, 

2nd  Edition,  University  of  California  Press  (1964). 

Yokokohji,  Y.,  and  T.  Yoshikawa,  “Bilateral  Control  of  Master-Slave  Manipulators  for 
Ideal  Kinesthetic  Coupling-Formulation  and  Experiment,”  Trans.  Robotics  and 
Automation,  Vol.  10,  No.  5,  pp.  605-620  (1994). 


